Explainability in Deep Reinforcement Learning

A large set of the explainable Artificial Intelligence (XAI) literature is emerging on feature relevance techniques to explain a deep neural network (DNN) output or explaining models that ingest image source data. However, assessing how XAI techniques can help understand models beyond classification tasks, e.g. for reinforcement learning (RL), has not been extensively studied. We review recent works in the direction to attain Explainable Reinforcement Learning (XRL), a relatively new subfield of Explainable Artificial Intelligence, intended to be used in general public applications, with diverse audiences, requiring ethical, responsible and trustable algorithms. In critical situations where it is essential to justify and explain the agent's behaviour, better explainability and interpretability of RL models could help gain scientific insight on the inner workings of what is still considered a black box. We evaluate mainly studies directly linking explainability to RL, and split these into two categories according to the way the explanations are generated: transparent algorithms and post-hoc explainaility. We also review the most prominent XAI works from the lenses of how they could potentially enlighten the further deployment of the latest advances in RL, in the demanding present and future of everyday problems.Comment: Article accepted at Knowledge-Based System

Explainability in Deep Reinforcement Learning

International audienceA large set of the explainable Artificial Intelligence (XAI) literature is emerging on feature relevance techniques to explain a deep neural network (DNN) output or explaining models that ingest image source data. However, assessing how XAI techniques can help understand models beyond classification tasks, e.g. for reinforcement learning (RL), has not been extensively studied. We review recent works in the direction to attain Explainable Reinforcement Learning (XRL), a relatively new subfield of Explainable Artificial Intelligence, intended to be used in general public applications, with diverse audiences, requiring ethical, responsible and trustable algorithms. In critical situations where it is essential to justify and explain the agent's behaviour, better explainability and interpretability of RL models could help gain scientific insight on the inner workings of what is still considered a black box. We evaluate mainly studies directly linking explainability to RL, and split these into two categories according to the way the explanations are generated: transparent algorithms and post-hoc explainaility. We also review the most prominent XAI works from the lenses of how they could potentially enlighten the further deployment of the latest advances in RL, in the demanding present and future of everyday problems

Explainability in Deep Reinforcement Learning

International audienceA large set of the explainable Artificial Intelligence (XAI) literature is emerging on feature relevance techniques to explain a deep neural network (DNN) output or explaining models that ingest image source data. However, assessing how XAI techniques can help understand models beyond classification tasks, e.g. for reinforcement learning (RL), has not been extensively studied. We review recent works in the direction to attain Explainable Reinforcement Learning (XRL), a relatively new subfield of Explainable Artificial Intelligence, intended to be used in general public applications, with diverse audiences, requiring ethical, responsible and trustable algorithms. In critical situations where it is essential to justify and explain the agent's behaviour, better explainability and interpretability of RL models could help gain scientific insight on the inner workings of what is still considered a black box. We evaluate mainly studies directly linking explainability to RL, and split these into two categories according to the way the explanations are generated: transparent algorithms and post-hoc explainaility. We also review the most prominent XAI works from the lenses of how they could potentially enlighten the further deployment of the latest advances in RL, in the demanding present and future of everyday problems

Collective eXplainable AI: Explaining Cooperative Strategies and Agent Contribution in Multiagent Reinforcement Learning with Shapley Values

International audienceWhile Explainable Artificial Intelligence (XAI) is increasingly expanding more areas of application, little has been applied to make deep Reinforcement Learning (RL) more comprehensible. As RL becomes ubiquitous and used in critical and general public applications, it is essential to develop methods that make it better understood and more interpretable. This study proposes a novel approach to explain cooperative strategies in multiagent RL using Shapley values, a game theory concept used in XAI that successfully explains the rationale behind decisions taken by Machine Learning algorithms. Through testing common assumptions of this technique in two cooperation-centered socially challenging multi-agent environments environments, this article argues that Shapley values are a pertinent way to evaluate the contribution of players in a cooperative multi-agent RL context. To palliate the high overhead of this method, Shapley values are approximated using Monte Carlo sampling. Experimental results on Multiagent Particle and Sequential Social Dilemmas show that Shapley values succeed at estimating the contribution of each agent. These results could have implications that go beyond games in economics, (e.g., for non-discriminatory decision making, ethical and responsible AI-derived decisions or policy making under fairness constraints). They also expose how Shapley values only give general explanations about a model and cannot explain a single run, episode nor justify precise actions taken by agents. Future work should focus on addressing these critical aspects

The toxicity of an “artificial” amyloid is related to how it interacts with membranes

Despite intensive research into how amyloid structures can impair cellular viability, the molecular nature of these toxic species and the cellular mechanisms involved are not clearly defined and may differ from one disease to another. We systematically analyzed, in Saccharomyces cerevisiae, genes that increase the toxicity of an amyloid (M8), previously selected in yeast on the sole basis of its cellular toxicity (and consequently qualified as “artificial”). This genomic screening identified the Vps-C HOPS (homotypic vacuole fusion and protein sorting) complex as a key-player in amyloid toxicity. This finding led us to analyze further the phenotype induced by M8 expression. M8-expressing cells displayed an identical phenotype to vps mutants in terms of endocytosis, vacuolar morphology and salt sensitivity. The direct and specific interaction between M8 and lipids reinforces the role of membrane formation in toxicity due to M8. Together these findings suggest a model in which amyloid toxicity results from membrane fission

Multi-scale approach to biodiversity proxies of biological control service in European farmlands

Intensive agriculture has profoundly altered biodiversity and trophic relationships in agricultural landscapes, leading to the deterioration of many ecosystem services such as pollination or biological control. Information on which spatio-temporal factors are simultaneously affecting crop pests and their natural enemies is required to improve conservation biological control practices. We conducted a study in 80 winter wheat crop fields distributed in three regions of North-western Europe (Brittany, Hauts-de-France and Wallonia), along intra-regional gradients of landscape complexity. Five taxa of major crop pests (aphids and slugs) and natural enemies (spiders, carabids, and parasitoids) were sampled three times a year, for two consecutive years. We analysed the influence of regional (meteorology), landscape (structure in both the years n and n-1) and local factors (hedge or grass strip field boundaries, and distance to boundary) on the abundance and species richness of crop-dwelling organisms, as proxies of the service/disservice they provide. Firstly, there was higher biocontrol potential in areas with mild winter climatic conditions. Secondly, natural enemy communities were less diverse and had lower abundances in landscapes with high crop and wooded continuities (sum of interconnected crop or wood surfaces), contrary to slugs and aphids. Finally, field boundaries with grass strips were more favourable to spiders and carabids than boundaries formed by hedges, while the opposite was found for crop pests, with the latter being less abundant towards the centre of the fields. We also revealed temporal modulation—and sometimes reversion—of the impact of local elements on crop biodiversity. To some extent, these results are quite unexpected and cause controversy because they show that hedgerows and woodlots should not be the unique cornerstones of agro-ecological landscape design strategies. We point out that combining woody and grassy habitats to take full advantage of the features and ecosystem services they both provide (biological pest control, windbreak effect, soil stabilization) may promote sustainable agricultural ecosystems. It may be possible to both reduce pest pressure and promote natural enemies by accounting for taxa-specific antagonistic responses to multi-scale environmental characteristics